{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dcf87b32",
   "metadata": {},
   "source": [
    "# Caching\n",
    "LangChain provides an optional caching layer for chat models. This is useful for two reasons:\n",
    "\n",
    "It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times.\n",
    "It can speed up your application by reducing the number of API calls you make to the LLM provider.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5472a032",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.globals import set_llm_cache\n",
    "\n",
    "llm = ChatOpenAI()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "357b89a8",
   "metadata": {},
   "source": [
    "## In Memory Cache"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "113e719a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 17.7 ms, sys: 9.35 ms, total: 27.1 ms\n",
      "Wall time: 801 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"Sure, here's a classic one for you:\\n\\nWhy don't scientists trust atoms?\\n\\nBecause they make up everything!\""
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "from langchain.cache import InMemoryCache\n",
    "set_llm_cache(InMemoryCache())\n",
    "\n",
    "# The first time, it is not yet in cache, so it should take longer\n",
    "llm.predict(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a2121434",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.42 ms, sys: 419 µs, total: 1.83 ms\n",
      "Wall time: 1.83 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"Sure, here's a classic one for you:\\n\\nWhy don't scientists trust atoms?\\n\\nBecause they make up everything!\""
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# The second time it is, so it goes faster\n",
    "llm.predict(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b88ff8af",
   "metadata": {},
   "source": [
    "## SQLite Cache\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "99290ab4",
   "metadata": {},
   "outputs": [],
   "source": [
    "!rm .langchain.db"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "fe826c5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can do the same thing with a SQLite cache\n",
    "from langchain.cache import SQLiteCache\n",
    "\n",
    "set_llm_cache(SQLiteCache(database_path=\".langchain.db\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "eb558734",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 23.2 ms, sys: 17.8 ms, total: 40.9 ms\n",
      "Wall time: 592 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"Sure, here's a classic one for you:\\n\\nWhy don't scientists trust atoms?\\n\\nBecause they make up everything!\""
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# The first time, it is not yet in cache, so it should take longer\n",
    "llm.predict(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "497c7000",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 5.61 ms, sys: 22.5 ms, total: 28.1 ms\n",
      "Wall time: 47.5 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"Sure, here's a classic one for you:\\n\\nWhy don't scientists trust atoms?\\n\\nBecause they make up everything!\""
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# The second time it is, so it goes faster\n",
    "llm.predict(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33815d3f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
